Regret Optimization¶

Regret optimization is a technique to optimize a portfolio given that one is uncertain about future prospects. Since one must eventually settle for a single portfolio out of all the possible scenarios, the best portfolio in this case is determine by the one that will have the least regret. This will be elaborated further

In general, this is a 2-stage optimization. The algorithm works as such,

Generate several possible scenarios in the future
- For each scenario, give it a discrete probability of occurrence
- These probabilities must sum to 1 across all scenarios!
In the first stage, derive the optimal weights for the portfolio for each scenario
- If there are 5 scenarios, there will be 5 sets of these weights
- Thus a model with 5 scenarios and 3 assets will yield a 5 x 3 matrix
In the second stage, solve for the minimal regret portfolio, these time using the previous sets optimal weights
- You will receive the final set of of weights here (a 1 x 3 vector)

Regret¶

Regret is defined as the cost of choosing one portfolio (which is optimal for a scenario) when another different scenario occurs instead. Functionally, the simplest mathematical formuation is

\[D(R(w_s) - R(w_o))\]

where \(D\) is a distance function, \(R\) is the profit function, \(w_s\) is the optimal weights for scenario \(s\) and \(w_o\) is the “optimal” weights that was chosen. Thus to solve for the minimal regret function, the exact problem formulation is as listed below

\[\underset{w}{\min} \sum_s^S p_s D(R(w_s) - R(w))\]

where \(p_s\) is the probability of scenario \(s\) occurring.

Distance function¶

The distance function could be anything that makes sense. Some common examples include a linear function, absolute function or quadratic function. Different functions will penalise regret differently and lead to different outcomes. For example, if a portfolio that does not have a wide swings in terms of objectives is desired, a quadratic function will work better than a linear function.

\[\begin{split}\begin{align} \text{Assuming} \quad R(w) &\geq 1 \quad \forall w \in W \\ [R(w_s) - R(w_o)]^2 &\geq R(w_s) - R(w_o) \end{align}\end{split}\]

Linear Approximations¶

Suppose we have convex objective and constraints functions for the first stage, we alter the second stage optimization a little. The outcome will be similar but it will give another nice interpretation of the results which will be explained later. Ideally these functions should be strictly linear. However, in practice, the differences are usually negligible.

Suppose we want to maximize the returns of the portfolio subject to some CVaR constraints. For simplicity, the returns function will be \(R\) and CVaR constraint functions will be \(C\). Thus our first stage optimization will be

For every single scenario \(s\) in \(S\)

\[\begin{split}\underset{w_s}{\max} R(w_s) \\ \text{subject to} \\ C_s(w_s) \geq 0 \\ \sum_i w_{s, i} = 1 \\ 0 \leq w \leq 1 \quad \forall w \in w_s\end{split}\]

From this we would get

\[\begin{split}W = \begin{bmatrix} w_{1, 1} & w_{1, 2} & \dots & w_{1, n} \\ \vdots & \vdots & \ddots & \vdots \\ w_{s, 1} & w_{s, 2} & \dots & w_{s, n} \end{bmatrix}\end{split}\]

where \(s\) is the number of scenarios adn \(n\) is the number of assets. We would then tweak our second (Regret) optimization to

\[\begin{split}\underset{a}{\min} \sum_s^S p_s D(R(w_s) - R(W \cdot a)) \\ \text{subject to} \\ \sum_s^S a_s = 1\end{split}\]

The solution of the problem, \(a\), will represent the proportion of importance that is taken from each scenario. Suppose there are 3 scenarios - X, Y, Z and that the final proportion derived is [0.2, 0.3, 0.5]. This means that 20% of the weights are taken from X, 30% from Y and 50% from Z. In essence, it weights the importance of each scenario for the final outcome.

To get the final weights, simply do a dot product of \(W \cdot a\).

Example¶

We will run through the Regret Optimization using both the RegretOptimizer and PortfolioRegretOptimizer classes. The PortfolioRegretOptimizer is a helper class that has several common built-in problems within itself. Underneath the hood, it uses the RegretOptimizer for the same operations. The RegretOptimizer is the more flexible tool that is useful for modelling more exotic scenarios.

[1]:

import numpy as np
from muarch.calibrate import calibrate_data

from allopy import OptData, RegretOptimizer
from allopy.datasets import load_monte_carlo

[2]:

# Generate different scenarios
num_assets = 7
num_scenarios = 4
scenario_probability = [0.57, 0.1, 0.14, 0.19]

main_adjustments = np.array([
    [ 0.0061,  0.0601,  0.0466,  0.0051, -0.0066, -0.0013, -0.0026],
    [ 0.0642,  0.0537,  0.0818,  0.0713,  0.0177,  0.0099,  0.0116],
    [-0.0219, -0.0381, -0.0059,  0.0242, -0.0153,  0.0164, -0.001 ],
    [-0.0333, -0.0617, -0.0405, -0.0251,  0.0084,  0.0054, -0.0035]
])

cvar_adjustments = np.array([
    [-0.0436,  0.0586, -0.0081,  0.0051, -0.0078,  0.0135, -0.0081],
    [ 0.0662,  0.079 ,  0.0896,  0.0501, -0.0265, -0.0572,  0.0025],
    [-0.1192, -0.1701, -0.1143,  0.0736, -0.0831,  0.0134, -0.0047],
    [-0.1728, -0.257 , -0.1933, -0.1432,  0.0342,  0.0079,  0.003 ]
])


def make_scenarios(adjustments, truncate=False):
    scenarios = []
    for adj in adjustments:
        data = OptData(load_monte_carlo()[..., :num_assets], 'quarterly')

        if truncate:  # cut for CVaR
            data = data.cut_by_horizon(3)

        scenarios.append(data.calibrate_data(adj))

    return scenarios

main_scenarios = make_scenarios(main_adjustments)
cvar_scenarios = make_scenarios(cvar_adjustments, True)

[3]:

# objective and constraint functions

def make_max_returns_obj_fun(cube: OptData):
    def obj_fun(w):
        return 1e2 * cube.expected_return(w, True)

    return obj_fun


def make_cvar_constraint_fun(cube: OptData, limit: float):
    def cvar_fun(w):
        return 1e3 * (limit - cube.cvar(w, True, 5.0))

    return cvar_fun


# limits and bounds
lb = [0, 0, 0.13, 0.11, 0, 0.05, 0.04]
ub = [1, 0.18, 0.13, 0.11, 1, 0.05, 0.04]

cvar_limit = [-0.34, -0.253, -0.501, -0.562]

[4]:

# optimization model formulation and execution
opt = RegretOptimizer(num_assets, num_scenarios, scenario_probability, sum_to_1=True)
opt.set_bounds(lb, ub)

obj_funcs = []
constraint_funcs = []
for m, c, limit in zip(main_scenarios, cvar_scenarios, cvar_limit):
    obj_funcs.append(make_max_returns_obj_fun(m))
    constraint_funcs.append(make_cvar_constraint_fun(c, limit))

opt.set_max_objective(obj_funcs)
opt.add_inequality_constraint(constraint_funcs)

final_weights = opt.optimize()

You can get the summary of the results. The first table show the optimal weight for each scenario. The second shows the proportion of each scenario and is only available when the approx option is set to True. The final table shows the final optimal weights.

[5]:

opt.summary()

[5]:

	Scenario_1	Scenario_2	Scenario_3	Scenario_4
Asset_1	0.2499	0.3495	0.1003	0.0000
Asset_2	0.1800	0.0946	0.0000	0.0000
Asset_3	0.1300	0.1300	0.1300	0.1300
Asset_4	0.1100	0.1100	0.1100	0.1100
Asset_5	0.2401	0.2259	0.5697	0.6700
Asset_6	0.0500	0.0500	0.0500	0.0500
Asset_7	0.0400	0.0400	0.0400	0.0400

	Scenario	Proportion (%)
0	Scenario_1	68.5900
1	Scenario_2	0.0000
2	Scenario_3	0.0000
3	Scenario_4	47.5900

	Weight
Asset_1	0.1714
Asset_2	0.1235
Asset_3	0.1510
Asset_4	0.1278
Asset_5	0.4835
Asset_6	0.0581
Asset_7	0.0465

The PortfolioRegretOptimizer contains a number of common optimization regimes with respect to regret optimization. We can apply the same optimization we did with the maximize_returns() method.

[6]:

from allopy import PortfolioRegretOptimizer

opt = PortfolioRegretOptimizer(main_scenarios,
                               cvar_scenarios,
                               scenario_probability,
                               rebalance=True,
                               sum_to_1=True,
                               time_unit="quarterly")

opt.set_bounds(lb, ub)
opt.maximize_returns(max_cvar=cvar_limit)
opt.summary()

[6]:

	Scenario_1	Scenario_2	Scenario_3	Scenario_4
Asset_1	0.2499	0.3494	0.1003	0.0000
Asset_2	0.1800	0.0947	0.0000	0.0000
Asset_3	0.1300	0.1300	0.1300	0.1300
Asset_4	0.1100	0.1100	0.1100	0.1100
Asset_5	0.2401	0.2259	0.5697	0.6700
Asset_6	0.0500	0.0500	0.0500	0.0500
Asset_7	0.0400	0.0400	0.0400	0.0400

	Scenario	Proportion (%)
0	Scenario_1	68.5900
1	Scenario_2	0.0000
2	Scenario_3	0.0000
3	Scenario_4	47.5900

	Weight
Asset_1	0.1714
Asset_2	0.1235
Asset_3	0.1510
Asset_4	0.1278
Asset_5	0.4835
Asset_6	0.0581
Asset_7	0.0465